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Abstract 

This  paper  presents  a  simple  algorithm  which  converts  any 
context-free  grammar  (without  e-productions)  into  a  connectionist 
network  which  parses  strings  (of  arbitrary  but  fixed  maximum  length) 
in  the  language  defined  by  that  grammar.  The  network  is  fast  and 
deterministic.  Some  modifications  of  the  network  are  also  explored, 
including  parsing  near  misses,  disambiguating  and  learning  new 
productions  dynamically. 
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1.  Introduction 


My  goal  in  designing  a  connection^  parser  was  to  be  able  to  build,  in  a 
systematic  way,  for  any  context-free  grammar,  a  network  which  will  parse 
strings  in  that  grammar  (within  a  length  restriction).  The  network  must 
represent  the  parse  tree  for  the  input  when  finished,  and  must  clearly 
indicate  when  there  is  no  parse  Most  important,  l  wanted  the  network  to  be 
deterministic  and  completely  general.  My  goal  is  not  cognitive  modeling  per 
se,  but  to  provide  a  powerful  technique  which  could  prove  useful  for  natural 
language  understanding  and  other  connectiomst  applications.  Context-free 
grammars  have  proven  very  useful  to  Computer  Scientists.  The  existence  of  a 
fast,  simple  and  relatively  efficient  connectiomst  parser  may  well  be  of  some 
importance  to  Cognitive  Scientists  working  with  connectiomst  models 

Several  other  connectiomst  parsing  schemes  have  appeared  recently. 
Certain  ideas  can  be  found  in  all  of  them  (including  mine).  They  all  parse 
context-free  grammars,  using  individual  units  to  stand  for  the  terminal  and 
nonterminal  symbols.  Given  a  production  such  as  S  -»  NP  VP,  there  are 
excitatory  connections  from  NP  and  VP  nodes  to  S  nodes  which  can  provide 
bottom-up  evidence  for  the  presence  of  an  S,  as  well  as  excitatory 
connections  from  S  to  NP  and  VP  providing  top-down  feedback.  When  a 
parse  completes,  the  active  units  and  their  connections  form  a  structure 
isomorphic  to  the  parse  tree  for  the  input.  I  will  at  times  refer  to  units  in  a 
parsing  network  as  parents  or  children  accordingly. 

The  work  of  Cottrell  (1985)  and  Waltz  and  Pollack  (1985)  encompasses 
natural  language  understanding  in  a  much  broader  sense  than  syntactic 
structure  alone.  Waltz  and  Pollack  do  not  even  attempt  to  build  a  general 
purpose  parsing  network.  Their  network  is  custom  built  for  each  input 
Cottrell's  networks  are  more  general.  He  even  has  a  program  to  build  the 
networks  from  an  input  grammar.  Unfortunately,  the  network  does  not 
always  find  the  correct  parse.2 

In  both  models,  contradictory  interpretations  of  the  input  are  mutually 
inhibitory.  Only  one  should  be  on  when  the  network  stabalizes.  Which  one 
depends  on  how  much  semantic  and  syntactic  support  each  receives  from  the 
rest  of  the  network.  A  parse  proceeds  by  activating  the  input,  e  g  the  node 
for  "the"  in  position  one,  the  node  for  "man"  in  position  two,  etc.,  and 
letting  the  network  run  until  it  settles  into  a  consistent  configuration 
representing  the  parse  of  the  input  with  all  ambiguities  resolved 

The  work  of  Selman  and  Hirst  (1985)  is  nearer  my  own  in  ambition 
their  goal  is  a  general  purpose  parsing  scheme  which  will  perform  correctly 
for  any  input.  They  do  not  consider  outside  influences,  such  as  semantics 
They  use  mutually  inhibitory  binder  nodes  to  connect  a  nonterminal  node  to 
all  the  subtrees  it  might  dominate,  each  binder  representing  a  different 


'  ^or  an  in-^oduction  -.o  the  connecttorist  oaracigm,  see  retdman  ana  3ailard  ( '  982) 

'his  can  oe  an  advantage  Torn  rr>e  ytewoomt  or  cognitive  mooe'irg  f  the  errors  mace 
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production  of  the  nonterminal.  Likewise,  binder  nodes  connect  a  node  to  all 
nodes  which  mightdommate  it.  They  use  a  variation  of  the  Boltzmann 
machine  (Hinton  and  Sejnowski,  1983)  computational  scheme.  The  units 
representing  the  input  are  clamped  on.  The  others  execute  asynchronously, 
turning  on  or  off  probabilistically  based  on  how  much  excitation  and 
inhibition  they  are  receiving.  Simulated  annealing  (Kirkpatrick,  Gelatt,  and 
Vecchi,  1983)  is  used  to  settle  the  network  into  a  state  where  the  active  units 
are  mutually  reinforcing  to  a  large  degree.  The  best  possible  state  is  one 
which  represents  a  legal  parse.  In  order  to  reach  this  optimal  state  with  high 
probability,  the  network  must  settle  gradually.  Selman  and  Hirst  used  24,000 
updates  for  each  unit. 

My  network  is  deterministic,  fast,  guaranteed  to  work  for  all  inputs  of 
any  context-free  grammar,  and  conceptually  very  simple  It  is  a  simple,  exact 
connectiomst  solution  to  the  computational  problem  of  parsing 
Contradictory  parses  do  not  inhibit  each  other  (but  see  section  three),  all 
possible  parses  proceed  in  parallel.  This  requires  a  large  number  of  units  - 
typically  tens  or  hundreds  of  thousands  (see  below) 

The  remainder  of  this  paper  is  organized  into  four  parts  Section  two 
describes  the  network  in  detail,  giving  an  algorithm  for  generating  it  from  a 
given  CFG.  An  exact  analysis  of  the  network's  complexity  is  presented,  along 
with  some  ways  of  trimming  its  size.  Section  three  touches  on  some  methods 
of  disambiguating,  i.e.  choosing  a  unique  parse  tree  for  ambiguous  input 
Section  four  gives  a  cursory  account  of  parsing  near-miss  input,  i  e.  input 
which  is  almost  grammatical  Section  five  gives  a  detailed  description  of  how 
productions  can  be  learned  dynamically  in  some  circumstances.  These  latter 
sections  are  intended  to  explore  the  flexibility  of  the  parser  as  well  as  test 
some  conneuiomst  learning  techniques  They  do  not  provide  an  account  of 
language  acquisition. 

All  networks  have  been  implemented  using  the  Rochester  Connectiomst 
Simulator  (Fanty  &  Goddard,  1986).  Simulation  traces  are  included  below 

2  The  Network 

2.1  Structure  and  function 

The  strategy  used  closely  parallels  that  of  the  CYK  parser  (Hopcroft  & 
Ullman,  1979).  The  network  contains  units  representing  the  terminals  and 
nonterminals  of  the  grammar  (several  for  each,  in  fact)  as  well  as  match  units 
which  will  be  explained  below  The  units  are  best  thought  of  as  organized 
mto  a  table,  with  the  columns  representing  starting  positions  in  the  input 
string,  and  the  rows  representing  lengths.  There  is  a  unit  for  each  terminal 
symbol  in  each  position  of  row  one  There  is  a  unit  for  each  nonterminal  at 
every  position  m  the  table  (potentially;  see  section  2  2).  Terminal  units  are 
activated  by  some  ouiMde  source  and  represent  the  input  to  the  parser.  A 
nonterminal  unit  will  become  active  if  other  units  representing  the  right 
hand  side  of  one  of  its  productions,  and  having  appropriate  starting  positions 
and  lengths,  are  active.  The  parse  proceeds  in  a  bottom-up  fashion  Figure  1 
illustrates  the  parsing  of  the  string  aabbb  for  the  grammar  snown.  The 
terminal  symbols  are  activated  as  input.  The  A  unit  in  the  first  row  becomes 
active  because  of  input  from  the  a  unit  m  the  same  position  and  the 
production  A  -*a  TheAumtm(2,1)  -  row  two,  column  one  -  becomes  active 
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starting  position 


Figure  1 

Parsing  the  input  aabbb.  Only  the  '■elevant  terminal  ana  nonterminal  units  are  snown 


because  of  input  from  both  the  a  unit  in  (1 ,1)  and  the  A  unit  in  (1,2)  and  the 
production  A-»aA.  Similarly  for  the  B  units.  The  S  unit  becomes  active 
because  of  input  from  A  in  (2,1)  and  B  in  (3,3).  Because  there  is  an  active  start 
symbol  in  column  one  whose  length  is  the  same  as  the  length  of  the  input, 
the  input  is  accepted.  In  order  to  mark  the  end  of  the  input,  a  special  S  unit 
becomes  active  at  the  end  of  the  input  string. 

The  active  units  represent  the  parse  tree.  Of  course,  there  will  in  general 
be  many  units  which  become  active  but  don't  represent  a  final  parse  tree.  In 
the  above  parse,  there  will  be  an  active  A  unit  in  (1,1),  for  example.  This  will 
not  affect  the  ability  of  the  network  to  recognize,  but  a  second,  top-down, 
pass  of  activity  is  necessary  in  order  to  pick  out  only  those  units  which 
participate  in  a  complete  parse  -  if  the  string  is  ambiguous,  more  than  one 
parse  will  stay  active  (see  section  three  for  possible  modifications).  The  top- 
down  pass  works  as  follows:  when  an  active  unit  representing  the  start 
symbol  beginning  in  position  one  and  of  length  n  receives  input  from  a  S 
symbol  in  position  n  +  1 ,  it  becomes  hyperactive.  This  unit  is  the  root  node  of 
all  parse  trees.  It  passes  this  hyperactivity  down  to  all  units  which  form  part 
of  one  of  its  completely  recognized  productions  They  pass  the  activity  down 
in  turn  until  it  reaches  the  bottom.  Only  the  hyperactive  units  represent  a 
parse.  In  order  to  better  distinguish  the  two  levels  of  activity,  a  unit  activated 
during  the  first  bottom-uD  pass  will  be  called  primed,  and  a  unit  active  after 
the  the  final  pass  will  be  called  on.  In  our  simulations,  primed  units  have  a 
potential  and  output  of  5,  on  units  have  a  potential  and  output  of  10 

A  more  detailed  account  of  how  the  network  works  follows  There  are 
three  kinds  of  units:  nonterminal  units,  terminal  units  and  match  units.  Each 
unit  has  two  sites,  or  locations,  where  an  incoming  connection  might  be 
made  One  is  for  bottom-up  input.  Enough  input  to  this  site  will  prime  the 
unit.  The  other  site  s  for  too-down  input.  Every  pair  of  units  with  a  bottom- 
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up  link  between  them  aiso  has  a  top-down  link,  if  a  primed  match  or 
nonterminal  unit  receives  input  to  the  top-down  site  from  an  on  unit,  't  will 
turn  on.  The  site  functions  are  described  below,  and  are  given  exactly  in 
appendix  two. 

The  terminal  units  are  the  simplest.  They  are  primed  by  some  outside 
source  and  represent  the  input  to  the  parser  They  are  all  on  row  one 
because  they  must  be  of  length  one.  One  per  starting  position  should  be 
primed  in  order  to  represent  an  input,  although  more  than  one  might  be 
activated  in  the  case  of  input  ambiguity  (see  section  three).  The  S  units  are  a 
special  kind  of  terminal  unit.  One  should  be  turned  on  in  the  position 
following  the  input 


-igure  2 

A  1  2  is  the  nonterminal  un>t  representing  A,  starting  at  position  i ,  and  of  ength  2  "nere  s 
a  matcn  unit  for  each  combination  of  lengths  tor  each  product, on 

The  match  units  are  used  to  reoresent  the  various  instances  of  productions 
for  nonterminal  units.  They  receive  bottom-up  inputs  from  units 
representing  the  symbols  on  the  r  h  s  of  their  oroouction  The  starting 
positions  and  lengths  of  these  units  must  be  consistent  with  the  nonterminal 
unit  bemg  served  and  with  each  other  For  each  nonterminal  unit  there  is 
one  match  unit  for  eacn  allowao'e  combination  of  iengtns  for  each 
production  (see -'gure  2)  The  bottom-up 'inks  to  the  match  units  are 
•we'ghted  so  tnat  all  the  connected  units  must  be  primed  before  the  matcn 
unit  becomes  primed  'he  bottom  uo  inputs  to  a  match  unit  are  processed 
with  a  filtered -sum  function  The  sum  of  the  inputs  is  taken,  but  each  in  out 'S 


allowed  to  contribute  at  most  some  fixed  amount.  This  seems  the  most 
natural  and  flexible  way  of  fixing  the  following  problem. 

Suppose  match  unit  X  represents  a  production  with  three  nonterminals  on 
the  r.h.s.  Two  of  the  three  nonterminal  units  are  primed,  giving  an  input  to 
the  unit  of  (with  weights  of  2/3  on  the  links) 

■j  2  2 

.5  *  -  -  5  *  -  -  i)  *  -  =  H.67 
3  3  3 

which  is  below  the  threshold  of  ten,  as  desired.  Now  suppose  the  first 
nonterminal  unit  turns  on  because  of  top-down  activation  from  somewhere 
else.  The  input  to  the  match  unit  is  now 

i  •>  •> 

1 0  *  -  -  5  *  —  -  I)  *  -  =  1 0 
3  .3  3 

which  causes  it  to  become  primed  falsely.  This  is  prevented  by  limiting  each 
bottom-up  input's  influence  to  5  *  weight. 

A  nonterminal  unit  receives  bottom-up  input  from  its  match  units  If  any 
of  them  become  primed,  this  means  one  of  the  productions  has  been 
realized,  so  the  nonterminal  unit  becomes  primed  A  nonterminal  unit 
responds  very  simply  to  bottom-up  input  from  its  match  units  if  any  are 
primed,  it  becomes  primed  and  provides  bottom-up  input  to  match  units 
above  it 

If  the  priming  eventually  reaches  a  start-symbol  nonterminal  unit  m 
column  one,  row  n  and  the  input  is  of  length  n,  then  there  is  a  parse  of  the 
input  This  root  unit  will  be  receiving  "top-down"  on  input  from  the  S  unit  in 
row  n  +  1  used  to  mark  the  end  of  the  input.  This  causes  it  to  turn  on.  It  now 
provides  top-down  on  input  to  its  match  units.  Any  which  were  primed  turn 
on,  and  provide  top-down  on  feedback  to  all  connected  units.  In  this  way  the 
parse  tree(s)  will  be  turned  on  from  the  top  down.  In  order  to  achieve  correct 
behavior,  the  units  must  respond  to  top-down  input  in  the  following  way.  If 
the  unit  is  already  primed  and  the  top-down  input  is  at  the  on  level,  then  the 
unit  turns  on.  If  the  unit  were  not  required  to  first  be  primed,  then  an  on 
nonterminal  unit  would  turn  on  all  its  match  units,  even  though  the 
production  instance  they  represent  may  be  quiet.  Match  units  have  only  one 
input  to  their  top-down  site,  so  they  can  simply  respond  to  it  in  a  thresholded 
manner.  Nonterminal  units  will  typically  have  many  inputs  to  their  top-down 
site.  Simply  summing  these  inputs  would  result  in  incorrect  behavior,  as  input 
from  two  primed  match  units  will  have  the  same  sum  as  input  from  a  Single 
on  match  unit.  To  differentiate  the  two  cases,  nonterminal  units  take  the 
maximum  input  as  the  value  of  the  too-down  site.  This  must  be  on  to  nave 
any  effect. 

When  primed  terminal  units  receive  on  input  at  their  top-down  site,  they 
turn  on  as  well.  It  is  possible  for  the  networic  to  detect  when  the  oarse  is 
complete  by  sensing  the  presence  of  an  on  terminal  symbol  in  every  column 
up  to  the  length  of  the  input,  if  we  exclude  productions  with  a  r.h.s  of 


length  one,  the  parse  will  complete  in  at  most  4*input-length  steps  The 
network  could  reject  input  by  timing  out. 

The  network  can  be  turned  off  by  taking  away  the  external  input  to  the 
terminal  units  (including  S).  This  will  remove  bottom-up  and  top-down 
activation  from  the  network:  the  source  of  all  bottom-up  activation  is  the 
terminal  units,  the  source  of  all  top-down  activation  is  the  S  unit  acting 
through  the  root  node. 

2.2  Network  construction 

A  program  to  build  a  network  for  a  given  context-free  grammar  has  been 
written  in  LISP  and  tested  using  the  Rochester  Connection^  Simulator  (Fanty, 
forthcoming).  The  algorithm  aDpears  in  the  appendix  in  a  pseudo  language 
The  strategy  is  to  work  bottom  up,  first  placing  each  terminal  in  each  position 
of  row  one.  Since  the  productions  of  a  unit  on  row  n  depend  only  on  units  m 
rows  n,  it  is  possible  to  build  the  network  in  a  single  bottom-up  pass 
througn  the  table.  For  each  combination  of  lengths  of  each  proa uct ion,  the 
appropriate  units  are  looked  up  m  the  table.  If  they  exist,  a  match  unit  for 
that  production  is  created  and  the  appropriate  links  made.  If  none  of  the 
productions  of  a  nonterminal  in  some  location  are  possible,  the  nonterminal 
unit  is  not  made  If,  for  example,  the  nonterminal  B  could  only  generate 
strings  of  length  three  or  more,  then  no  nonterminal  units  of  the  form  B  n  1 
or  B  n  2  would  be  created,  and  there  would  be  no  match  units  in  charge  of 
productions  looking  for  such  units. 

e-productions  are  not  allowed,  which  is  not  too  limiting,  as  a  grammar 
with  e  productions  can  always  be  translated  to  one  without  They  could 
easily  be  added  if  desired.  If  there  are  productions  of  the  form  X-*  Y.  where  Y 
is  a  single  nonterminal,  then  Y  units  must  be  processed  before  X  units  on  each 
row. 

2.3  Complexity  of  the  network 

In  order  to  facilitate  the  discussion,  I  introduce  the  following 

L  =  maximum  length  of  input  string 

T  =  set  of  terminals  in  the  grammar 

.V  =  set  of  nonterminals  in  the  grammar 

m  n)  =  set  of  productions  of  nonterminal  n 

v ( pi  =  list  of  nonterminals  m  production  p  (may  be  repeats) 

rip)  =  list  of  terminals  m  production  p  (may  oe  reoeats) 

The  number  of  nonterminal  and  terminal  units  is  reasonable  in  the  worst 
case  there  are 

Lit.  -  1  i 

-  .V  -  LT 

of  them.  The  number  of  match  units  is  significantly  larger,  however  in  the 
worst  case  there  are 


match  units.  The  sum  represents,  for  each  row,  for  each  column,  for  eacn 
nonterminal  at  that  location,  for  all  productions  of  that  nonterminal,  all  the 
possible  combinations  of  constituent  lengths  in  an  expansion  of  that 
production.  The  quantity  (r-l)-jr(p)|  choose  |v(p)|-l  represents  the  number  of 
different  assignments  of  lengths  to  constituents  which  will  sum  to  the  desired 
length.  The  reasoning  is  as  follows  How  many  ways  can  three  constituents 
sum  to  ten7  There  are  two  boundaries  between  constituents  to  be  chosen, 
and  nine  different  locations  to  choose  from:  (10-1)  choose  (3-1).  The  term 
rip, '|  appears  in  the  first  term  because  terminals  reduce  by  one  the  space  into 
which  the  nonterminals  may  expand. 

For  example,  if  L  =15,  N  =  10,  |n(n)|  =  5  and  |v(p.i|  =  3  for  all 
nonterminals  and  ignoring  terminals,  then  the  network  will  have  at  most 
1 ,200  nonterminal  units  and  1 53,750  match  units.  This  figure  can  best  be 
improved  be  keeping  v(p)  small  (by  putting  the  grammar  in  Chomsky  normal 
form,  for  example).  If  |v(p)|  =  2  m  the  above  example,  the  number  of  matcn 
units  will  be  at  most  34,000  The  size  of  the  network  is  0(nm  +  ’),  where  n  is 
the  length  of  the  network  and  m  is  the  number  of  nonterminals  on  the  right- 
hand  side  of  the  productions.  Thus,  it  may  be  desirable  in  practice  to  limit  the 
number  of  nonterminals  on  the  right-hand  side  of  a  production  to  two  if  this 
is  done,  the  size  of  the  network  will  be  0(n3). 

For  the  following  grammar,  taken  from  Selman  and  Hirst  (1985),  for 
inputs  of  length  up  to  15,  the  numberofnon  terminal  umtsis570and  the 
number  of  match  units  is  2,040. 


S  -  NP  VP 

NP  -  determiner  NP2 

S  -  VP 

NP  -  NP2 

VP  verb 

NP  -  NP  PP 

VP  -  verb  NP 

NP2  -►  noun 

VP  -*  VP  PP 

PP  -  preposition  NP 

NP2  -  adjective  NP2 

The  number  of  units  may  be  significantly  smaller  than  the  worst  case  if 
several  nonterminal  units  cannot  generate  strings  of  all  lengths  (esoecially 
short  lengths).  The  network  described  above  would  have  30  more 
nonterminal  units  and  210  more  match  units  if  PPs  could  generate  strings  of 
length  one.  A  better  example  of  the  potential  savings  is  prov'deo  be  the 
following  grammar  which  has  the  same  number  of  nonterminals  ana 
productions  as  the  preceeding  grammar,  but  no  nonterminal  can  generate 
strings  of  length  one. 


For  inputs  oflength  up  to  15,  thenumberofnon  terminal  units  is  458  and  tne 
number  of  match  units  is  1  561  if  nonterminal  units  for  each  nonterminal 
were  olaced  at  each  location  of  the  table,  the  number  of  nonterminal  units 
would  be  600  and  the  numberofmatch  units  would  be  2794. 

The  simulations  run  in  O(n)  time,  where  n  is  the  length  of  the  input 
Multiplying  the  execution  time  oy  the  number  of  units  (with  the  length  of  the 
r  n.s.  of  productions  limited  to  two),  we  get  a  total  of  0(na)  computation 
steps.  The  serial  execution  time  for  parsing  is  O(nT)  for  straightforward 
parsing  algorithms  and  about  0(n^  5)  for  the  asymptotically  best  algorithm  so 
far  This  means  that  the  network  is  bigger  slower  than  the  best  we  could 
expect  by  about  a  factor  of  n.  This  is  because  the  algorithm  s  not  compie'ety 
parallel izable  The  activity  must  work  its  way  up  from  the  bottom  ser  ally 

2.4  Implementation  using  two-state  linear  threshold  units 

It  is  possible  to  build  an  equivalent  network  using  only  simoie,  single-site, 
'inear  threshold  units  which  have  only  two  levels  of  activity  on  ana  off  f  1  and 
0)  The  transformation  is  simple.  Replace  every  unit  m  the  original  networx 
with  a  pairofumts-one  for  the  bottom-up  pass,  and  one  for  to  the  top- 
down  pass  (see  Figure  3).  The  bottom-up  unit  being  active  corresponds  to  me 
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’wo  simQ-er  units  car  do  tne  :oo  ot  ore  -rcre  comoiex  unit 


original  unit  bemg  pnmed\  the  too-down  unit  being  active  corresponds  to 
the  original  unit  bemg  on.  The  bottom-up  inputs  to  match  unit  pairs  w 1 1  be 
weighted  as  oefore  to  require  them  ail  to  oe  on  They  no  'oncer  need  to  oe 
filtered  as  they  only  come  from  other  bottom-uo  units  The  top  a  own  unit 
does  not  neea  to  take  the  maximum  of  its  nouts  as  it  only  receives  input  from 
otner  top  down  nodes  m  order  to  require  that  the  pair  be  primec  before 
fuming  on  the  weiqnt  on  the  link  from  the  bottom-up  unit  to  the  toe  down 
jnu  neeos  to  equal  the  sum  of  the  weights  on  tne  other  m puts  to  *he  too 
oown  unit,  anq  the  threshold  on  the  top-Oowri  unit  must  oe  set  so  mat  t 
must  receive  mout  from  the  bottom-uo  unit  and  at  least  one  too-oown  ,nout 


When  the  parse  completes,  the  active  top-down  units  represent  the  parse 
tree. 

2.5  Informal  proof  of  correctness 

In  order  to  establish  the  correctness  of  a  network  as  described  above,  we 
need  only  show  three  things.  (Assume  the  grammar  has  no  e-productions.  All 
units  begin  off  except  those  terminal  units  representing  the  input,  which  are 
primed.) 

(1)  A  nonterminal  unit  A.m.n  will  become  primed  if  and  only  if  other 
terminal  and  nonterminal  units  spanning  m  through  m  +  n  in  exactly  the 
order  of  one  of  As  productions  are  primed. 

We  will  say  that  these  other  terminal  and  nonterminal  units  satisfy  one 
of  A's  productions.  A  will  become  primed  if  and  only  if  one  of  its  match 
units  becomes  primed.  These  are  the  only  units  with  connections  to  its 
bottom  site,  and  only  input  to  this  site  will  cause  an  off  unit  to  become 
primed.  Each  match  unit  corresponds  to  one  of  A's  productions.  It  has  an 
input  from  a  unit  corresponding  to  each  symbol  in  the  production  in  such 
a  way  that  positions  m  through  m  +  n  are  spanned  exactly.  The  weights  on 
the  inputs  to  the  match  units  are  such  that  they  must  all  be  primed  in 
order  for  the  input  to  exceed  the  threshold.  Each  input  is  filtered  so  that 
no  single  input  can  contribute  more  its  share.  Every  possible  satisfaction 
of  each  of  A's  productions  has  a  corresponding  match  unit. 

(2)  When  the  input  is  of  length  n,  the  first  nonterminal  unit  to  turn  on  (if  any 
do)  will  be  S.  1. n,  and  it  will  turn  on  only  if  it  is  first  primed.  (S  is  the  start 
symbol.) 

In  order  for  a  nonterminal  unit  to  turn  on  it  must  first  be  primed  and 
then  receive  on  input  to  the  top-down  site  Since  input  from  an  on  unit  is 
required  to  turn  another  unit  on,  the  first  unit  to  turn  on  must  receive  top- 
down  input  from  S.n  +  1.1,  the  only  unit  on  from  the  beginning.  The  only 
unit  with  such  a  connection  is  S.1  n 

(3)  A  nonterminal  unit  -  other  than  the  first  to  turn  on,  as  in  (2)  -  will  turn  on 
if  and  only  if  it  was  first  primed  and  it  is  one  of  the  units  satisfying  a 
nonterminal  unit  which  turned  on  previously. 

Following  (2),  we  need  only  show  that  a  primed  nonterminal  unit  will 
receive  input  to  its  top-down  site  from  an  on  unit  if  and  only  if  it  helps 
satisfy  some  production  of  an  on  nonterminal  unit.  Except  for  input  from 
the  on  S  unit,  which  has  just  one  connection  to  the  root  node, 
nonterminal  units  receive  input  to  their  top-down  site  only  from  match 
units  to  which  they  contribute.  A  match  unit  will  turn  on  just  when  it  was 
first  primed  (production  satisfied)  and  its  parent  nonterminal  unit  turns 
on.  All  inputs  to  a  primed  match  unit  contribute  to  the  satisfaction  of  its 
production  instance. 

2.6  Simulation  results 

A  network  for  the  grammar  m  section  2.3  was  built  and  simulated  with 
the  following  input:  det  noun  verb  det  adj  adt  noun,  which  corresponds  to 


sentences  such  as  The  man  kissed  the  tall  attractive  woman  The  resuits  of  the 
simulation  are  given  in  table  1 .  Only  units  which  have  non-zero  potential  are 
shown.  Match  units  are  omitted  m  order  to  make  the  table  more  readable. 
After  26  steps,  the  network  is  stable.  Those  units  with  a  potential  of  10 
represent  the  (unique)  correct  parse. 


Table  1 

Simulation  of  det  noun  verb  det  adj  ad /  noun 

Unit  name 

Potential  after  1 
step 

Potential  after  1 3 
steps 

Potential  after  26 
steps 

det  1 

5 

5 

10 

noun  2 

5 

5 

10 

verb  3 

5 

5 

10 

det  4 

5 

5 

10 

ad]  5 

5 

5 

10 

adj  6 

5 

5 

10 

noun  7 

5 

5 

10 

S  8 

5 

5 

5 

NP2.2.1 

0 

5 

10 

NP  2  1 

0 

5 

5 

VP  3  1 

0 

5 

5 

S3  1 

0 

5 

5 

NP2  7  1 

0 

5 

10 

NP  7  1 

0 

5 

5 

NP  1  2 

0 

5 

10 

S.2  2 

0 

5 

5 

NP2  6  2 

0 

5 

10 

NP.6  2 

0 

5 

5 

S.1  3 

0 

5 

5 

NP2.5  3 

0 

5 

10 

NP  5  3 

0 

5 

5 

NP  4  4 

0 

5 

10 

VP  3  5 

0 

5 

10 

S  3  5 

0 

5 

5 

S  2.6 

0 

5 

5 

S  1  7 

0 

5 

10 

Table  2  shows  the  results  of  simulating  noun  vero  det  noun  preo  noun 
prep  det  noun,  e  g  John  hit  the  man  with  Tom  with  a  hammer  This  sentence 
is  ambiguous  in  many  ways  Notice  the  overlapping  constituents,  such  as  the 
PP  from  5  to  9  (PP  5  51  and  the  NP  ‘rom  3  to  6  (NP.3  4)  The  match  nodes 
provide  enough  information  to  distinguish  the  various  oarses,  but  if  the 
match  nodes  are  invisible  externally,  the  state  of  the  networ<  does  not  make 


'0 
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Table  2. 

Simulation  of  noun  verb  det  noun  prep  noun  prep  det  noun 
(terminals  not  shown) 


Unit  name 


NP2.1.1 
NP.1.1 
VP  2.1 
S  2.1 
NP2.4.1 
NP.4.1 
NP2.6.1 
NP  6.1 
NP2  9  1 
NP  9  2 


Potential  after  1 3 
steps 


S  1.2 
P.3  2 
P  5.2 
P  8  2 


VP  2  3 
S  2  3 
NP  4.3 
PP.7.3 


S  1  4 
NP  3.4 
NP  6.4 


VP  2.5 
$.2.5 
PP  5.5 


NP  3.7 


VP  2  8 
S  2.8 


Potential  after  26 
steps 

Potential  after  26 
steps  with 
disambiguation 
(section  threei 

10 

10 

10 

10 

5 

5 

5 

5 

10 

10 

5 

5 

10 

10 

10 

10 

10 

10 

5 

5 

5 

5 

10 

10 

10 

10 

10 

10 

10 

10 

5 

5 

5 

5 

10 

10 

5 

5 

10 

5 

10 

5 

10 

10 

5 

5 

10 

5 

5 

5 

5 

5 

10 

5 

10 

10 

5 

5 

10 

10 

3.  Disambiguating 

A  lot  of  local  disambiguating  happens  naturally  because  some 
interpretations  do  not  participate  m  a  complete  parse  tree  and,  thus,  never 
turn  on.  This  could  account  for  word  sense  disambiguation  in  many  cases 
For  example,  to  parse  the  sentence  The  man  walked  on  the  deck,  both  noun 
and  verb  would  be  primed  in  position  six,  but  only  noun  would  turn  on,  as 
there  is  no  complete  parse  using  the  other  interpretation 

When  the  input  is  truly  syntactically  ambiguous,  however,  more  than  one 
parse  tree  will  be  on  simultaneously  The  parse  can  be  made  unambiguous  by 
allowing  only  one  match  node  (i  e  production)  per  nonterminal  unit  to 
remain  active.  The  simplest  way  to  do  this  is  to  order  the  match  units  of  each 
nonterminal  and  add  inhibiting  links  from  each  to  those  of  lesser  rank.  Any 
inhibiting  input  from  a  superior  match  unit  would  be  enough  to  prevent 
activation  Only  the  highest  ranking  primed  unit  would  stay  pnmea  The 
match  units  would  need  to  be  ranked  not  only  according  to  which  production 
of  the  grammar  they  represent,  but  also  according  to  the  combination  of 
lengths  of  their  production.  The  ranking  would  not  need  to  be  consistent 
throughout  the  network.  One  production  could  dominate  another  only  for 
short  lengths  or  towards  the  beg  inn  mg, for  example 

The  above  scheme  was  not  implemented;  however,  a  different  scheme 
was.  In  this  scheme,  each  match  unit  inhibits  all  the  other  match  units 
belonging  to  the  same  nonterminal  node  An  offmatch  unit  receiving 
inhibiting  input  will  not  become  primed  Thus,  this  scheme  prefers  shallower 
parse  trees. 

Because  the  simulator  used  is  synchronous,  multiple  match  units  will  prime 
simultaneously  whenever  the  subtrees  have  equal  depth  When  this 
happens,  the  inhibition  must  be  gradual,  or  the  match  units  will  turn  each 
other  off  This  will  allow  them  ail  to  come  back  on  the  following  step  and 
cycle  in  this  manner  indefinitely.  The  following  behavior  reliably  yields  a 
single  winner  The  inhibiting  weights  between  the  match  units  vary 
randomly  between  -0  5  and  -1 .0  exclusive.  A  primed  unit  receiving  inhibition 
will  lower  its  potential  by  an  amount  equal  to  the  strongest  inhibiting  input 
When  the  potential  gets  to  zero,  it  turns  off.  When  only  one  match  unit  is 
left,  the  lack  of  inhibition  allows  it  to  gain  its  full  primed  potential  of  five 
For  example,  suppose  two  match  units  become  primed  at  the  same  time  and 
that  match  unit  M 1  inhibits  M2  with  weight  -  0.6  and  M2  inhibits  M  1  with 
weight  -  0.65.  The  following  is  a  trace  of  their  behavior  m  one  step 
increments  (with  minor  arithmetic  errors): 

Ml  5  0  1.75  0.45  0.0  0  0 

M2  5  0  199  0  94  0  67  5  0 

M2  has  a  higher  potential  after  the  first  round  of  inhibition  because  it  is 
more  weakly  inhibited  Now  M2  is  receiving  inhibition  equal  to  -  0  6*  1  75 
and  Ml  is  receiving  inhibition  equaito  -  0  65  *  1  99  M2 's domination  of  Ml 
s  increasing.  There  is  no  way  for  M 1  to  push  M2  under  0  since  its  potential  s 
less  than  M2's  and  its  inhibition  is  less  than  its  potential  With  more  than  two 
match  units  on  the  interactions  are  more  complicated.  If  the  inhibiting  inputs 
were  summed,  it  would  be  possible  for  every  match  unit  to  go  off  after  one 
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step  and  oscillate  as  described  above.  This  is  why  the  total  inhibition  is  equal 
to  the  strongest  single  input. 

If  the  inhibitory  weights  between  match  units  were  adjusted  dynamically 
according  to  how  often  the  match  unit  came  on,  then  the  network  could 
learn  to  prefer  more  common  interpretations.  It  might  also  be  possible  to  use 
nonsyntactic  information  to  affect  the  preferred  parse  with  external 
contributions  to  the  inhibition.  This  was  not  implemented. 

The  last  column  of  Table  two  shows  a  simulation  of  the  ambiguous 
sentence  from  the  previous  section  using  the  disambiguation  scheme  just 
described.  A  single  unambiguous  parse  tree  results  No  match  units  battled, 
as  the  ambiguities  involved  parse  trees  of  different  depths,  the  first  on  won 
by  default.  For  example,  in  deciding  between  the  two  productions  VP  -  verb 
NP  and  VP  -  VP  PP,  the  latter  always  wins  because  its  parse  tree  is  shallower 
I  make  no  claims  about  the  adequacy  of  this  scheme.  I  present  it  as  a 
demonstration  of  how  disambiguation  could  be  done  in  this  model.  Other 
strategies  are  possible  as  well. 

4.  Parsing  near-miss  input 

It  is  sometimes  desirable  to  give  a  reasonable  parse  of  input  strings  which 
are  not  i n  the  language  defined  by  the  grammar  -  ungrammatical  but  still 
understandable  natural  language  utterances,  for  example  The  parsing 
network  described  above  is  too  rigid  to  do  this  effectively  if  some  of  the  input 
is  missing  or  there  is  extra  input.  But  if  the  source  of  the  ungrammatically  is 
simply  the  substitution  of  incorrect  input  of  the  same  length  as  some  correct 
mput,  then  it  can  be  made  to  do  this  by  having  match  units  become  partially 
primed  to  an  extent  reflecting  the  closeness  of  the  match  between  the  input 
string  and  the  production. 
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implemented  a  network  that  extends  the  disambiguating  networx 
described  previously  Match  units  whose  productions  are  partially  satisfied 


may  have  a  potential  between  0  and  2  This  range  was  chosen  so  that 
inhibition  from  a  partially-primed  match  unit  will  always  be  less  than 
inhibition  from  a  primed  match  unit.  The  weights  on  inhibiting  connections 
range  from  -  .5  to  -  1  0.  A  primed  match  unit  will  always  deliver  inhibition 
of  -  2  5  or  less;  a  partially-primed  match  unit  will  always  deliver  inhibition  of 
-  2  0  or  more.  Primed  match  units  always  inhibit  partially-primed  match 
units  Partially-primed  match  units  compete  with  each  other  much  as  primed 
match  units  do  (see  section  three),  except  the  inhibition  from  a  partially 
onmed  match  unit  is  not  strong  enough  to  prevent  the  priming  of  other 
match  units  This  is  necessary,  as  a  match  unit  may  become  partially  primed 
before  one  of  its  brothers  becomes  fully  pnmed  The  strongest  partially- 
primed  match  unit  may  be  beaten  out  by  weaker  partially-primed  match 
units  because  of  the  random  variation  m  the  strength  of  the  inhibiting 
weights 

3ecause  inhibition  from  a  partially-primed  match  unit  does  not  prevent 
other  match  units  from  becoming  partially-primed,  care  must  be  taken  to 
prevent  the  re -priming  of  a  partially-primed  match  unit  which  has  just  been 
defeated  Unless  it  subsequently  receives  additional  input  and  becomes  fully 
primed,  a  defeated  oartiaily-primed  match  unit  will  stay  off. 

The  permissiveness  of  the  network  can  be  adjusted  by  setting  a  minimum 
input  required  for  partial  priming.  In  the  simulations  run,  the  cutoff  was  3 
(an  input  of  10  represented  complete  satisfaction  of  the  production)  The 
potential  of  a  partially-primed  match  unit  is  equai  to  its  total  input  divided 
by  5  Some  of  the  key  steps  m  a  simulation  of  the  ungrammatical  input  det 
det  noun  verb  noun  are  given  m  Table  three  The  grammar  used  was  the  one 


Table  3 

Near-miss  parse  of  det  det  noun  verb 
(Only  relevant  units  shown.) 

noun 

Potential  after  steD 

Unit 

0 

|  3 

:  7 

i 

10 

14 

15 

17  | 

j 

23  and  after 

det  1 

5  0 

1  5  0 

;  5 

5  0 

5  0 

5  0 

5  o  ! 

10  0 

det  2 

5  0 

1  5  0 

5  0 

5  0 

5  0 

5  0 

5  o  ; 

5  0 

noun. 3 

5  0 

!  5  0 

5  0 

5  0 

5  0 

5  0 

5  0  1 

10  0 

verb  4 

5  0 

!  5  0 

i  5  0 

5  0 

5  0 

5  0 

5  0  ! 

10  0 

noun  5 

5  0 

:  5  o 

5  0 

5  0 

5  0 

5  0 

5  0  1 

10  0 

S  6 

5  0 

5  0 

5  00 

5  0 

10  0 

10  0 

10  0 

10  0 

NP2  3  1 

0  0 

:  5  0 

5  0 

5  0 

5  0 

5  0 

5  0 

10  0 

NP2  5  1 

0  0 

5  0 

5  0 

5  0 

5  0 

5  0 

5  0 

10  0 

NP  5  1 

0  0 

0  0 

5  0 

50 

5  0 

5  0 

5  0 

10  0 

NP2  2  2 

0  0 

i  o  o 

1  05 

1  05 

1  05 

1  05 

1  05 

10  0 

VP  4  2 

0  0 

1  05 

5  0 

5  0 

5  0 

5  0 

10  0 

10  0 

NP  1  3 

0  0 

■  1  05 

1  27 

1  27 

1  27 

1  27 

10  0 

10  0 

S  1  5 

0  0 

’  0  0 

0  0 

1.32 

1  32 

10  0 

10  0  ■ 

10  0 

m  section  2.3.  Only  the  relevant  units  are  included.  As  before,  many  units  not 
m  a  complete  parse  will  become  primed.  Even  more  become  partially- 
primed.  All  the  non-match  units  which  eventually  turn  on  are  shown.  The 
partial-priming  of  NP.  1 .3  after  step  3  is  due  to  input  from  det.  1 ,  after  step  7, 
it  (actually,  its  match  unit)  is  also  receiving  some  input  from  the  partially- 
primed  NP2.2.2  so  its  potential  increases  to  127.  Eventually,  the  partial 
priming  reaches  the  root  node  S.  1  5  What  happens  next  differs  from 
previous  networks.  In  this  network,  the  end-marker,  $.6,  must  have  potential 
equal  to  10  in  order  to  provide  sufficient  top-down  feedback  to  turn  on  the 
root  node.  It  requires  five  steps  to  reach  that  level.  This  is  to  give  complete 
parses  a  chance  to  reach  the  top.  Once  a  partially-primed  match  node  turns 
on,  it  is  too  late  for  a  fully  primed  match  node  to  inhibit  it.  When  the  parse 
completes,  the  section  of  the  input  which  does  not  fit  into  the  parse  remains 
primed. 

In  a  simulation  with  input  det  det  det  verb  det  det  det,  no  parse 
completed  NP1 .3  and  VP4.4  were  both  partially  primed,  but  their  combined 
input  was  less  than  the  threshold  for  S.  1 ,7's  match  unit. 

5.  Learning  new  productions  dynamically 

The  near-miss  network  described  above  has  been  modified  to  learn  new 
productions  dynamically.  The  circumstances  under  which  it  is  capable  of 
learning  are  depicted  in  Figure  5.  After  a  near-miss  parse,  there  will  be  a  gap 


ctgu re  5 

When  the  cmaiiC'ion  A -B  C  s  earned  ;he  ojrse  w  r  be  con^oie' e 

m  the  parse  tree  where  some  constituent  was  "expected"  but  not  founo  if 
the  gap  can  be  parsed  as  one  or  two  constituents,  then  a  match  noae 
representing  the  new  production  will  be  recruited  (Feldman,  1982)  (A  -*8  C  m 
Figure  5). 


This  mechanism  cannot  explain  the  acquisition  of  a  grammar  from  raw 
data.  For  one  thing,  no  new  nonterminals  are  learned.  This  is  especially 
limiting  given  the  restriction  on  production  length  (see  below),  it  can 
sometimes  account  for  new  rules  composed  of  known  constituents  The  reai 
purpose  of  this  section  is  to  explore  the  flexibility  of  the  parsing  network  l 
do  not  claim  to  have  an  adequate  mechanism  for  grammar  acquisition 

5.1  Local  learning 

This  section  describes  the  recruitment  of  a  match  node  to  represent  a 
production  instance  The  production  will  not  be  recognized  elsewhere  m  the 
network.  In  order  to  make  the  network  more  tractable,  the  right-hand  side 
of  productions  must  be  of  length  one  or  two  I  will  call  such  productions  type 
one  and  two  respectively.  For  any  context-free  grammar,  there  is  a  weaxly 
equivalent  grammar  satisfying  this  restriction  Each  bottom-up  input  to  a 
match  unit  now  goes  to  its  own  site  Accompanying  each  nonterminal  unit  is 
a  single  learn  unit  and  some  free  match  units  which  do  not  yet  represent  a 
production  Fixed  match  units  represent  production  instances  as  before 
except  for  the  presence  of  two  bottom-up  sites  for  type  two  productions  A 
nonterminal  unit  being  turned  on  from  above  without  having  first  been 
primed  means  an  instance  of  the  nonterminal  is  expected  but  not  found  This 
turns  the  learn  unit  on,  which  enables  the  free  match  units  If  the  input  in 
question  can  be  parsed  as  one  or  two  constituents,  then  some  match  unit  will 
be  recruited  to  represent  this  production  instance. 

The  additional  bottom-up  sites  are  to  enable  free  match  units  to  detect 
when  they  are  receiving  bottom-up  input  from  a  potential  production  Each 
free  match  unit  can  learn  only  production  instances  with  some  fixed 
combination  of  constituent  lengths  or  division  For  example,  the  free  match 
unit  in  Figure  6  can  learn  productions  with  two  constituents,  the  first  of 
length  four  and  the  second  of  length  six.  Notice  that  any  combination  of  one 
input  to  the  bottom -right  site  and  one  input  to  the  bottom -left  site 
constitutes  a  legal  production. 

Figure  6  depicts  the  setup  of  a  free  match  unit  just  before  learning  occurs 
The  free  match  unit  is  receiving  bottom-up  input  to  each  site  from  exactly 
one  unit  It  has  not  yet  primed  because  it  is  in  a  free  state;  it  requires 
additional  input  from  the  learn  unit  before  responding  The  nonterminal 
unit  B  3  10  is  on  but  no  match  unit  is  primed  This  will  cause  the  learn  unit  to 
come  on  The  learn  unit  requires  on  input  from  8  3  10  and  s  inhibited  by  the 
match  units  Once  the  learn  unit  comes  on,  the  free  match  unit  will  become 
highly  active  briefly  I  wii!  call  this  state  excited  It  now  inhibits  the  learn 
unit,  whose  job  is  done.  Ifmorethanoneinputper  bottom -up  site  had  been 
active  the  free  match  unit  would  not  have  responded 

Three  changes  now  occur  which  transform  the  free  match  umt  into  a  fixea 
match  unit  First,  the  unit  no  ion ger  enters  the  free  state  m  wmcn  input  from 
the  learn  unit  is  required  Second,  the  weights  on  all  inactive  bottom  - uo  l<n<s 
are  zeroed  The  match  unit  now  responds  only  to  the  pair  .earned  Third,  the 
top-down  links  to  the  learned  constituents  are  given  positive  weights  It  is 
the  need  for  this  weight  change  which  necessitates  the  special  excited  activity 
of  a  match  unit  which  has  just  learned  a  production  Weight  change  occurs  at 
the  destination  unit.  The  only  way  for  B  3  4  and  C7  6  (see  Agure  6)  to  <now 
to  mcrease  the  we'ght  on  the  top-acwn  1 1 n ks  from  the  matcn  unit  s  from  this 


earn  site 


zero  weight 


B.3.4 


B.7.6 


Configuration  of  free  match  unit  just  before  learning  The  Learn  unit  >s  about  to  turn  on 


level  of  activity  The  other,  non-pnmed  units  receiving  top-down  excited 
input  do  not  change  the  weights.  Because  of  the  excited  input,  B .3.10,  B. 3  4 
and  C.7  6  become  momentarily  excited  as  well.  Soon,  all  excited  units  settle 
to  an  on  state  and  the  parse  completes.  The  situation  after  learning  is  shown 
in  Figure  7. 

It  is  possible  for  free  match  nodes  in  separate  divisions  to  each  learn  a 
production  at  the  same  time.  This  ability  is  necessary  for  global  learning  in 
the  next  section.  Since  all  match  nodes  inhibit  each  other,  it  is  necessary  to 
suppress  this  inhibition  during  learning.  Because  of  this,  all  interpretations  of 
ambiguous  input  will  be  learned  if  they  are  in  different  divisions  and  none 
will  be  learned  if  they  are  m  the  same  division.  If  there  is  more  than  one 
match  node  representing  a  division,  only  one  should  learn  a  new  production 
This  can  be  achieved  by  ordering  the  free  match  nodes  of  a  division  and 
putting  strong  inhibitory  connections  from  the  earlier  match  nodes  to  the 
learn  site  of  the  later  ones. 

Table  four  shows  a  simulation  of  a  local -learning  network  for  the  simple 
grammar 


S  -  A  B 


B  -  b 


The  production  instance  being  learned  is  B.3  1  -  b.  1 .  After  step  i  3,  the  near 
miss  parse  has  activated  B  3. 1  even  though  it  was  never  primed  The  learn 
unit  and  the  free  match  unit  take  a  couple  of  steps  to  come  on  B  3  1  and, 
after  steQ  1  5,  the  learn  unit  are  decaying.  If  learning  does  not  haDoen,  they 
will  turn  off  eventually  Notice  that  a. 3  becomes  excited  two  steos  after  the 
match  unit  does.  The  delay  >s  caused  by  the  need  to  increase  the  we'gnt  on 
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Table  5 

Same  input  as  in  Table  4,  but  after  learning 
(Only  relevant  units  shown.) 


Unit 

Potential  after  step 

6 

7 

9 

10 

1 1 

a.  1 

5 

5 

5 

5 

10 

a. 2 

5 

5 

5 

5 

10 

a. 3 

5 

5 

5 

5 

10 

$  4 

10 

10 

10 

10 

10 

B  3  1 

5 

5 

10 

10 

10 

learnB  3  1 

0 

0 

0 

0 

0 

match 

5 

5 

5 

10 

10 

A. 1.2 

5 

5 

10 

10 

10 

S.1.3 

5 

10 

10 

10 

10 

the  link  from  the  match  node  to  a. 3.  Table  five  highlights  a  parse  of  the  same 
input  after  learning 

5.2  Global  learning 

This  section  describes  an  extension  to  the  above  scheme  which  distributes 
a  production  instance  learned  locally  throughout  the  whole  network 
Although  the  exact  mechanism  isdifferent,  the  idea  of  using  a  central 
template  to  program  other  representations  was  insQired  by  McClelland's 
(1985)  Connection  Information  Distributer  I  will  first  give  a  high-level 
description  of  what  happens,  then  provide  a  more  detailed  description  of  the 
implemented  network.  There  is  a  single,  global  representation  of  each 
production  When  learning  occurs  locally,  the  production  learned  is  noted  in 
the  global  store.  After  the  network  calms  down,  it  enters  a  special  learn 
state,  which  limits  the  spreading  activation.  When  global  learning  occurs,  the 
units  involved  in  the  production  are  activated  throughout  the  network  by  the 
global  representation  in  such  a  way  that  the  local  learning  mechanism  of  the 
previous  section  burns  in  all  the  production  instances. 

When  every  instance  of  some  production  in  the  network  is  active  and 
learning  simultaneously,  a  great  deal  of  care  must  be  taken  to  insure  cross¬ 
talk  does  not  occur.  A  single  unit,  such  as  NP.3.4,  may  be  both  the  parent  and 
son  for  the  same  production  In  order  to  distinguish  the  various  roles  a  unit 
may  play,  a  separate  unit  is  used  for  each  To  facilitate  this,  the  grammar 
must  be  in  Chomsky  normal  form,  which  means  that  all  productions  have  the 
form 

X  -  YZorX  -a,  where  X,  Y  and  Z  are  nonterminals  and  a  is  a  terminal  Any 
context-free  grammar  can  be  converted  to  a  weakly  equivalent  grammar  in 
Chomsky  normal  form  (Hopcroft  &  Ullman,  1979).  For  sucn  a  grammar,  there 
are  only  three  roles  a  nonterminal  can  olay  parent,  left-son  and  right-son. 
Accordingly,  the  joo  previously  done  by  nonterminal  units  .s  now  done  by  a 
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Figure  8 

0roauction  instance  A34-»B31C43  with  relevant  '.nks  snown 


trio  of  units,  one  for  each  role  of  the  nonterminal.  Figure  8  shows  the  setuo 
for  the  production  instance  A  3  4  -»  8  3. 1  C  4.3. 

Normally,  the  three  nonterminal  units  pass  on  activation  Bottom-uD 
activation  is  spread  from  the  match  unit  to  the  parent  unit  to  the  two  son 
units  which  are  connected  to  other  match  units.  Top-down  activation  comes 
from  one  of  the  son  units  and  goes  through  the  parent  unit  to  the  match 
unit.  However,  the  spread  of  activation  between  parent  and  son  units  for  a 
nonterminal  group  is  blocked  when  the  network  s  m  a  global  learn  state  in 
the  networks  implemented,  this  i$  done  by  having  the  activation  go  through 
pass  units  which  are  mhibited  by  a  Globa  Ilea  rn  Unit  (see  Figure  10)  When 
the  network  is  not  m  a  global  learn  state,  it  behaves  just  like  the  one 
described  in  section  5  1 ,  disambiguation,  near-rmss  parsing  and  locai  learning 
work  the  same  way  The  network  has  a  central  representation  of 
productions  comprised  of  four  pools  of  units  (Figure  9)  The  too  pooi  has  one 
unit  for  every  nonterminal  and  represents  that  nonterminal  as  a  parent  The 
bottom  left  (bottom  right!  pool  also  has  one  unit  for  every  nonterminal  and 
represents  that  nonterminal  when  it  is  the  left-son  (right-son)  The  termma. 
pool  has  one  unit  oer  terminal  and  represents  that  terminal  as  a  son  To 
represent  the  production  A  -*B  C,  a  production  unit  will  have  two  wav 
excitatory  links  to  the  A  unit  in  the  top  pool,  the  B  unit  m  the  bottom  left 
pool,  and  the  C  unit  in  the  bottom  right  pool 

~he  global  units  are  connected  to  nonterminal  groups  tnrougnout  me 
parsing  network  m  the  following  way  (see  Figure  1 0 )  Each  unit  in  the  too 
pool  is  connected  to  the  top  site  of  the  oarent  unit  in  a  l  the  nontermma' 
groups  for  that  't.  When  a  unit  in  the  too  pooi  turns  or  the  oottom  units 
m  ail  the  corresponding  nonterminal  groups  turn  on  Eaco  un  *  n  the  oottom 
left  pool  is  connected  to  the  oottom  site  of  toe  eft  son  uni:  m  ail  tne 
nonterminal  grouos  for  that  un  t  When  a  unit  n  the  bottom  eft  pod  turns 
on.  the  top  eft  units  m  ail  the  corresoonomg  nonterminal  groups  oecome 
primeo  There  are  similar  connect'ons  *rom  tne  bottom  ngnt  oooi  to  the 
right-son  unit  of  eac.n  cor-esoonamg  n0uterm, na  grouo  and  from  the 
terminal  pool  to  the  termma1  un.ts 
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Giooai  orcaucnon  'emoiates 


-gure  -C 

Nonterminal  unit  reoiaceo  bv  sece~  jo  ts  n  g  odoi  ejrp,-g  re’wo'1 
Samoie  connect  ors  Tom  g  ooai  urns  are  spow' 


For  each  connection  descnoed  in  the  preceding  paragraph,  there  .s  a 
•  reciorocal  connection  to  the  units  m  the  central  pools  if  a  unit  m  a 

‘V  nonterminal  group  becomes  excited,  it  will  cause  tne  corresponding  unit  in 

the  central  pool  to  become  armed  If  exactly  one  unit  in  the  top  0001  ano 
:  in  one  in  each  oottom  or  one  m  the  terminal  pool  are  prmed,  a  unique 

oroduction  >s  represented  by  the  pattern  of  activation  This  is  how  the  global 
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production  representation  knows  wnen  local  learning  has  occurred  and  what 
has  oeen  iearned 

Here  is  what  happens  when  learning  occurs.  The  local  learning 
mechanism  causes  some  production  to  be  learned.  The  parent  and  two 
children  of  the  production  instance  learned  become  excited  In  the  parent 
nonterminal  group,  only  the  bottom  unit  becomes  excited  because  the  pass 
units  m  a  nonterminal  group  do  not  pass  on  excitation  Similarly,  m  the  left 
son  nonterminal  group,  only  the  upper  left  unit  becomes  excited,  and 
likewise  for  the  right  son  This  causes  the  corresponding  units  n  the  centra' 
pools  to  become  primed  (This  does  not  seriously  affect  the  parsing  network  ) 
The  global  production  unit  representing  this  production  is  now  receiving 
activation  from  ail  three  components  and  will  become  primed  after  two 
simulation  steos  If  more  than  one  global  production  unit  attempts  to 
become  primed  at  the  same  time,  mutual  inhibition  will  force  them  all  to  zero 
potential  and  global  learning  of  these  productions  will  not  occur.  This 
prevents  the  cross  talk  which  would  occur  if  two  productions  were  turned  on 
m  the  central  template  simultaneously.  Once  a  global  production  unit 
becomes  primed,  it  innibits  all  other  giobal  production  units.  The 
components  of  the  production  in  the  global  nonterminal  and  terminal  poois 
become  inactive  as  soon  as  they  stop  receiving  excited  input  from  the 
network  The  primed  feedback  from  the  global  production  unit  is  not 
enough  to  keep  them  active  Self  feedback  keeos  the  production  unit 
primed  it  will  not  turn  on  until  the  parse  is  finished.  It  requires  excitation 
from  the  GiobalLearnUnit,  which  does  not  turn  on  until  the  networc  has 
finished  a  parse  (during  simulations,  this  unit  is  turned  on  oy  n and  I 

When  the  parse  completes,  the  :ermmal  units  are  turned  off  and  the 
network  calms  down  The  Global  Learn  unit  is  turned  on  This  causes  the  shil 
primed  production  unit  to  turn  on,  wnicn  turns  on  the  global  units 
comprising  the  production  For  example,  if  the  production  to  oe  iearned  is 
A^BC,  then  the  A  unit  m  tne  too  pool,  tne  B  unit  m  the  lower  left  pooi,  and 
theC  unit  m  the  lower  r.ght  pool  will  be  on  This  causes  ail  bottom  units  m  A 
nonterminal  groups  to  turn  on  and  all  top  left  units  in  B  nontermma'  groups 
and  all  top  rig n t  units  m  C  nonterminal  groups  to  become  primed  Because 
the  GloballearnUrnt  s  inhibiting  the  pass  units,  activation  will  not  spread 
within  nonterminal  groups 

The  situation  isjustwhat  s  needed  for  local  learning  to  occur  all  over  the 
network  The  bottom  unit  of  the  B  nonterminal  groups  is  on,  but  no  fixed 
match  unit  is  onmed  For  each  division,  there  will  be  one  free  match  unit 
receiving  input  from  B  as  eft  son  and  C  as  right  son.  These  productions  will 
be  learned  (This  w 'll  not  affect  the  giooai  poois  of  units  in  thee  current 
state  )  After  a  few  steos,  the  giobai  units  become  exnausted  ana  turn  off. 
They  wiil  remain  quiescent,  for  a  few  steos  longer  -  enough  to  let  the  network 
calm  down  When  the  network  un  ts  lose  mput  ‘rom  these  giobai  units,  they 
quickly  die  down 

in  the  networx  descnoea  so  far  g  ooal  learmng  would  not  occur  at  the 
nonterminal  group  wnicn  learned  the  product  on  locally  For  example, 
suppose  the  local  production  instance  A  4  3-341  C52  s  learned  4  httie 
while  ater  global  earning  ta«es  place  8  4  2  and  C  6  1  are  onmed  and  A  4  3 
s  on  ,  out  A  4  3  will  not  ;earn  this  production  instance  oecause  3  4  1  and 
C  5  2  are  also  primec,  wmch  pr  m.es  rhe  now  fixed  match  unit  for  th-s 


production,  which  inhibits  the  local  learn  unitforA.4.3.  This  is  avoided  by 
having  the  Global  Learn  Unit  inhibit  the  inhibition  of  local  learn  units  It  has 
a  strong  positive  connection  to  all  local  learn  units  at  the  inhibit  site.  We  now 
must  worry  about  redundant  match  units  being  learned  This  is  prevented  by 
having  fixed  match  units  inhibit  free  match  units  with  the  same  division  If 
the  production  being  learned  is  already  known  for  some  division,  then  that 
fixed  match  unit  will  become  primed,  and  no  new  match  unit  will  be  fixed 
This  is  an  extension  of  a  mechanism  already  in  place:  free  match  units  already 
inhibit  other  free  match  units  (ranked  lower)  within  the  same  division.  These 
inhibiting  connections  remain  even  after  the  unit  becomes  fixed  All  we  need 
to  add  are  inhibiting  connections  from  match  units  which  begin  life  fixed. 
With  this  addition,  the  new  production  can  truly  be  learned  globally  It  also 
makes  the  network  more  robust,  as  described  in  the  next  section 

The  network  builder  was  programmed  to  make  two  unfixed  match  nodes 
per  division  per  nonterminal  group  in  order  to  test  that  no  redundant 
productions  were  learned.  Simulations  for  the  following  grammar  worked 
correctly  (the  results  are  somewhat  big  for  a  table,  so  they  will  be 
summarized): 

S  -*  A 8  A-»  a  C-*c 

B-BC  B-*b 

First,  the  network  was  run  with  the  input  b  c  a  b.  This  resulted  in  a  near-miss 
parse  (actually  the  miss  was  not  too  near).  The  production  instance 

A. 1.3-B  12A.3.1  was  learned.  This  caused  the  three  units  A.  1 .3  parent, 

B.  1  2. Ison  and  A. 3. 1  rson  to  become  excited,  which  primed  the  global  units 
A  Parent,  3  Lson  and  A. Rson,  which  primed  A->B.A,  the  global  production 
umtforA-BC.  A  couple  of  steps  later  the  excited  network  units  calmed 
down  to  on  This  caused  all  the  global  units  except  A->B  A  to  turn  off 
When  the  parse  completed,  the  terminal  units  were  turned  off  and  the 
network  was  run  until  only  A->B.A  was  left  orimed  (about  12  steps)  The 
Global  Learn  Unit  was  turned  on  by  hand,  which  turned  on  A- >B.  A,  which 
turned  on  A. Parent,  B  Lson  and  A. Rson.  On  input  from  these  units  turned  on 
all  A.x  y.  pa  rent  units  and  primed  all  B  x.y.  Ison  and  A.x  y  rson  units.  Learn 
units  fo  rail  the  A.  x.y  g  roups  came  on  The  ones  with  length  greaterthan  one 
had  free  match  units  get  excited,  one  per  division,  and  learn  their  new 
productions 

The  A  1 .3  group  had  its  fixed  match  unit  for  A.  1  3  -B  1  2  A  3  1  on  me 
because  of  the  production  instance  it  had  just  learned  This  did  notprevent 
the  learn  unit  from  coming  on  because  the  Global  Learn  Unit  was  on  it  did 
prevent  the  other  free  match  unit  with  the  division  2  +  1  from  activating  One 
free  match  unit  for  the  division  1  +  2  became  excited  and  learned  the 
production  instance  A.1.3-B  1,1  A.2.2.  After  a  few  steps,  the  global 
nonterminal  units  (but  not  the  Global  Learn  Unit)  become  exhausted  as  oo  all 
tne  learn  units.  The  network  quickly  calms  down. 

After  learning,  the  networx  was  tested  with  the  input  b  b  a  a  b,  whicn 
requires  requires  the  use  of  two  new  instances  of  the  production  A  -  B  A  The 
parse  comQleted  successfully  The  following  units  were  on  when  the  network 
stabilized 
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S  1  5  par 


A  1 .4  Ison 
A  1  4. par 

A  2  3  rson 
A  2  3  par 


8  2  2  Ison 
B  2  2  par 


B  1 . 1  Ison 

B  2  1  Ison 

C  3  1  rson 

A  4  1  rson 

3  5  1  rson 

8.1.1  par 

B  2. 1  par 

C  3  1  par 

A  4  1  par 

B  5  1  oar 

b  1 

b  2 

a  3 

a  4 

b  5 

5.3  Deferred  learning 

The  global  learning  described  in  the  previous  section  would  fail  to 
distribute  productions  learned  locally  if  more  than  one  is  learned  during  a 
parse.  It  is  possible  for  the  network  to  take  advantage  of  periods  of  quiet  to 
catch  up  on  unlearned  productions.  The  mechanism  I  propose  (but  have  not 
implemented)  is  spontaneous  excited  activity  of  fixed  match  units  when  the 
network  is  quiet.  The  probability  of  a  match  unit  becoming  excited  would  be 
inversely  proportional  to  the  length  of  time  it  has  been  fixed,  so  that  recently 
fixed  match  units  would  be  more  likely  to  become  excited.  The  excited  match 
unit  would  excite  the  parent  and  children  of  the  production,  which  would 
start  global  learning  as  in  the  previous  section  More  than  one  match  unit 
activating  simultaneously  would  do  no  harm.  Global  learning  of  already 
learned  productions  wiil  have  no  effect  as  described  above 

One  way  to  accomplish  this  would  be  to  run  a  link  from  the  global!  earn 
unit  (which  could  just  as  easily  be  a  network  of  units)  to  match  units  The  link 
would  be  to  a  new  site.  The  global  learn  unit  is  on  when  parsing  is  not  taking 
place,  so  if  the  match  unit  responded  to  input  to  this  site  by  becoming  excited 
probabilistically,  we  would  have  the  desired  effect  The  weight  on  this  link 
could  be  made  non-zero  when  the  unit  becomes  fixed  and  gradually  decay 
after  that.  Recently  fixed  match  units  would  get  the  most  activation  and, 
thus  be  more  likely  to  fire. 

Rehearsing  productions  also  makes  the  network  more  robust  If  a 
match  unit  were  to  "die",  then  another  would  be  recruited  to  ta«e  its  place 
the  next  time  its  production  was  rehearsed. 

6.  Discussion 

I  consider  the  major  advantage  of  my  parser  to  be  its  generality  and  its 
quick,  sure  results  The  major  difference  oetween  it  and  other  Darsmg 
schemes  is  the  way  it  maintains  all  possible  parses  in  parallel,  eliminating  tne 
need  for  search  (i  e.  relaxation)  One  of  the  goals  of  connectionism  :s  to 
account  for  the  solution  of  complex  tasks  m  a  few  computational  steps  using 
massive  oarallehsm.  My  network  ooes  exactly  that. 
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The  major  disadvantage  of  my  parser  is  its  rigid  structure  and  fixed 
length.  Because  the  length  of  the  network  is  fixed,  the  set  of  strings  parsed  is 
finite.  Of  course,  this  is  true  of  any  implemented  parsing  mechanism,  but  one 
might  hope  for  an  extendable  structure  It  would  be  nice  if  the  network 
could  parse  longer  strings  by  acquiring  more  resources  (i  e.  units)  on  the  fly. 
McClelland's  (1985)  CID  mechanism  may  prove  useful  in  this  capacity 
(McClelland  and  Kawamoto,  1986),  though  the  resource  requirements  of  CID 
are  substantial.  I  would  very  much  like  to  be  able  to  efficiently  acquire  and 
release  general  purpose  working  units  as  needed  Nevertheless,  I  think  that 
fixed  length  structures  can  prove  useful  and  may  point  the  way  to  flexible 
structures  which  perform  the  same  task. 

The  parsing  algorithm  upon  which  my  network  is  based  is  an  example  of 
dynamic  programming  (Aho,  Hopcroft  &  Ullman,  1974)  Other  problems  with 
efficient  dynamic  programming  solutions  may  have  similar  parallel 
implementations.  One  such  problem  is  finding  the  number  of  edit  operations 
( insert ,  replace  and  delete)  required  to  convert  one  string  to  another.  A  fast 
implementation  of  this  algorithm  could  be  very  useful  in  cognitive  tasks 
requiring  pattern  matching.  The  values  passed  during  a  computation  are  the 
number  of  edit  steps  so  far.  This  means  that  a  potentially  large  amount  of 
information  must  be  comumcated,  unlike  the  parsing  network  in  which  only 
the  existence  of  constituents  is  communicated.  While  this  does  not  stand  in 
the  way  of  a  fast  parallel  implementation,  it  may  present  complications  for  a 
connectiomst  model,  where  the  output  values  of  units  are  not  meant  to  carry 
much  information  (Feldman  and  Ballard,  1982). 

The  learning  mechanism  described  is  certainly  inadequate  as  a  theory  of 
language  acquisition,  but  in  keeping  with  the  purpose  of  the  paper,  itdoes 
demonstrate  some  techniques  which  may  prove  useful.  In  connectiomst 
models,  multiple  copies  of  subnetworks  which  do  the  same  task  are  common 
Learning  m  such  a  way  that  all  copies  are  kept  consistent  is  a  difficult 
problem  My  solution  does  this  with  a  minimum  of  overhead.  It  may  even  be 
possible  to  use  a  similar  mechanism  to  learn  new  syntactic  categories 
dynamically.  The  learning  results  also  demonstrate  that  the  network  can  be 
flexible 

There  are  several  directions  future  work  might  take.  One  would  be  to 
make  the  length  restriction  more  flexible.  Another  would  be  to  improve  the 
learning  algorithm.  The  results  given  only  scratch  the  surface  of  what  is 
needed  Another  we  have  partially  worked  out  would  be  to  efficiently 
extend  the  network  to  handle  augmented  grammars  (Gazdar  et  a  I . ,  1985)  in 
which  the  grammar  symbols  have  properties.  Rules  are  applicable  only  if  the 
properties  meet  the  accompanying  restrictions,  e  g.  that  the  NP  and  VP  have 
the  same  number. 
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Appendix  One 

Network  Building  Algorithm 

length  is  the  maximum  length  of  the  input  string 
nonterminals  is  the  set  of  nonterminals 
terminals  is  the  set  of  terminals 
start-symbol  is  the  start  symbol  of  the  grammar 

for  i  *—  1  to  length  /*  make  the  terminal  units  V 

for-each  term  in  terminals 

MakeUmt(  type  «—  Terminal,  name  <—  term.i.  1) 

endfor-each 

MakeU n i t(ty pe  *—  Terminal,  name  «—$./.  1) 

endfor 

for  row  «—  1  to  length 

for  col  < —  1  to  1  +-  ( length  -  row) 
for-each  nt  in  nonterminals 

for-each  prod  in  Productions-of(nt) 

if  (row  LengthOf(prod))  then  /*  possibly  room  * 
DoProduci\on(row,col,nt,prod) 
endif 

endfor-each 

endfor-each 

endfor 

endfor 


/*  connect  end  markers  with  start  symbol  units  */ 
for  i  *—  1  to  length  -  1 

if  (Exists(sfarf-symoo/.  1 ./ ))  then 

MakeLink(from  *—  S  i  +  1.1 ,  to  «—  starx-symool  1  /, 
weight  *—  2,  site  <—  too) 

endif 

endfor 

if  ( Ex\sts(start-symbol .  1  length  ))  then 

MakeUnit(type  «—  Terminal,  name  «—  $  length  +  l  ' ) 
Makelmk(from  «—  S  length  +  1  1,  to  <—  start-sympoi  '  length, 
weight  <—  2,  site  «—  top) 

endif 

DoProduct\on(length,start,nt,prod)  <— 

*  The  vectors  ntlen  and  sum  are  used  to  generate  every  possmie 

'*  combination  of  lengths  of  symbols  in  th'S  production  ^ r / e o { , ] 

/*  the  length  of  the  T"  nonterminal  m  proa  sum[i\  sthesumof 
/*  the  lengths  of  the  first  /  -  1  nonterminals  m  proa 

*  set  up  initial  configuration  *. 

weight*—  2  0  LengthOf(prod) 
pieces  <—  NumberOfNontermmals(proa) 


nttot «—  length  -  NumberOfTerminals(prod)  ,*  length  ofall  nt's  * ' 


*  check  for  an  all -terminal  production  of  the  wrong  length  * ' 
if  pieces  =  0  and  length  -  LengthOf(prod)  then  return 

for  i  < —  1  to  pieces  -  / 
ntlen[i ]  *—  1 
sum[i ] «—  /  -  1 

endfor 

sum[p/eces] «—  pieces  -  1 
ntlenlpieces)  <—  nffof  -  sum[p/eces] 

.  *  loop  once  for  each  configuration  */ 

loop  *  until  break  */ 

*  test  configuration  '/ 

where  <—  start  ■  *  where  the  next  symbol  must  begin  ' ' 
whichnt  <r-  i  /*  which  nonterminal  is  next  in  the  prod  ’ 
for-each  symbol  in  prod 

if  lsNontermmal(symbo/)  then 

if  Exists  (symbol. where  ntlen[whichnt})  then 
whichnt  <—  whichnt  +  1 
where  <—  where  +•  ntlen[which] 
else  goto  next  /*  this  configuration  fails  v 
endif 

else  '*  symbol  is  a  terminal  * 
if  Exists(symbor  where  1 )  then 
where  <—  where  +  1 

else  goto  next  ' *  this  configuration  fails  * 

endif 

endif 

endfor-each 


i*  a  configuration  with  all  subordinate  units  in  existence 
/*  has  been  found  * 

if  Not(Exists(nf  start. length))  then 

MakeUnit(type  «—  Nonterminal,  name  <—  nt. start. length) 

endif 

*  match  units  are  not  named,  so  the  mdex  of  the  made 
,  *  unit  is  saved  m  a  variable  for  future  reference 
match  MakeUnitftype  «—  Match) 

where  start  '  where  the  next  symbol  must  begin  * 
whichnt  *— 1  *  which  nonterminal  is  next  in  the  prod  * 

*  ma<e  links  between  match  and  subordinate  units  * 
for-each  symbol  in  prod 

if  lsNontermmal(symbo/)  then 

MaKetmk(from  <—  symbol. where. ntlen[whichnt\, 

to  <—  match,  site  bottom,  weight «—  weight ) 
MaKeLinkfto  «—  sympol  where. ntlen[whichnt], 
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from  *—  match,  site  <—  top,  weight «—  1 ) 
whichnt  <—  whichnt  +  1 
where  *—  where  +  ntlen[which ] 
else  /*  symbol  is  a  terminal  */ 

MakeLink(from  <—  symbol. where. ntlen[whichnt], 

to  <—  match,  site  <—  bottom,  weight  <—  weigh 
MakeLinkfto  *—  symbol. where. ntlen[whichnt], 

from  <—  match,  site  *—  top,  weight «—  1 ) 
where  <—  where  +  1 
endif 
endfor 

/*  make  links  between  match  and  nt  *, 

MakeLmkffrom  «—  match,  to  *—  nt. start. length,  site  *—  bottom, 
weight  «—  t) 

Makelmkfto  <—  march, from  «—  nt. start. length,  site  <—  top, 
weight  <—  1 ) 

next:  /*  make  next  configuration  */ 

change  *—  pieces  -  1 
whilefchange  >  0  and 

ntlen[change ]  +  sum[change ]  ??  nttot  -  ( pieces  -  change)) 
change  *—  change  -  1 

endwhile 

d{change  <  1)then  exitloop  /*  all  done  -  no  more  */ 

/*  configurations  *' 

endif 

ntlen[change ]  <—  nf/en(change)  +  1 
for  i  change  +  1  to  pieces 
ntlen[i ]  <—  1 

sum  <—  sum[i  -  1  ]  +  ntlen[i  -  1  ] 

endfor 

ntlen[pieces]  *—  nttot  -  sum[pieces ]  /*  last  must  take  up  slack 

endloop 


,•  V  v  *-•  v  . 

>  W. .  A.  .X.  ».  Jl_  lA.  .  A_ '.rw  .  A.  .JW  -JW  . 


Appendix  Two 

Unit  Functions  for  Simple  Network 


*  site  functions  -  these  return  a  value  for  each  site  of  the  unit  These  values 
are  used  by  the  other  functions  below  *' 

*  SFsum  is  used  by  bottom  site  of  nonterminal  units  */ 

SFsum(/npuf-//sr) 

sum  =  0  0 

for-each  input  in  mput-hst 

sum  =  sum  +  (Value(mpuf)*Weight(/npu  0) 

endfor-each 
return  sum 
end  SFsum 

*  SFfilterSum  is  used  by  bottom  site  of  match  units  * ' 

SFfilterSum(mput-hst) 

sum  =  00 

for-each  input  in  mput-hst 

sum  =  sum  +  Mm(ValueOnpuf),  5  0)  *  Weight(/npuf)) 

endfor-each 
return  sum 

end  SFfilterSum 

*  SFmax  is  used  by  top  site  of  nonterminal  and  terminal  units  * 
SFmax(mput-hst) 

hold  =  0  0 

for-each  input  in  mput-hst 

if  (hold  <  (Value(inpuf)  *Weight(/npur)))  then 
hold  =  (Value(inpuf)  *Weight(/nput)) 

endif 

endfor-each 
return  sum 
end  SFsum 

'  *  These  functions  are  called  to  set  unit  parameters  after  the  site  functions 
have  been  called  The  same  one  can  be  used  for  all  units  it  external  primed 
input  is  provided  for  the  input  string  * 

UFparse(unif) 

\f(umt  state  =  off  and  SiteValue(unif, "bottom")  10  0)  then 
unit  state  =  pnmed 
unit  potential  =  50 
umt  output  =  5 

else  \f{umt  state  =  primed  and  SiteValuefumr.  "top")  ’0  0)  then 
unit  state  =  on 
unit  potential  =  10  0 
unit  output  =  1 0 
endif  -*  else  leave  it  alone  * 
end  UFparse 


